A Random Forest Guided Tour

Tree Prediction

Each tree estimates the response at point \(x\) as:

\[ m_n(x; \Theta_j, D_n) = \frac{\sum_{i \in D_n(\Theta_j)} \mathbf{1}_{X_i \in A_n(x; \Theta_j, D_n)} Y_i}{N_n(x; \Theta_j, D_n)} \]

Forest Prediction

The forest estimate for \(M\) trees is:

\[ m_{M, n}(x) = \frac{1}{M} \sum_{j=1}^{M} m_n(x; \Theta_j, D_n) \]

Random Forest Regression

Splitting Criteria:

• The CART-split criterion is used to find the best cut applying:

Stopping Condition:

Nodes are not split if they contain fewer than nodesize points or if all \(X_i\) in the node are identical.

Prediction:

\[ m_{M, n}(x) = \frac{1}{M} \sum_{j=1}^{M} m_n(x; \Theta_j, D_n) \]

Random Forest Classification

Splitting Criteria:

• The Gini impurity measure is used to determine the best split:

\[ G = 1 - \sum_{k=1}^{K} p_k^2 \]

Prediction:

• Each tree makes a prediction using the majority class in the cell containing \(x\).
• Classification uses a majority vote:

\[ m_{M, n}(x; \Theta_1, \ldots, \Theta_M, D_n) = \begin{cases} 1 & \text{if } \frac{1}{M} \sum_{j=1}^{M} m_n(x; \Theta_j, D_n) > \frac{1}{2} \\ 0 & \text{otherwise} \end{cases} \]

Like this…

[3]

The Data

1. Where the data came from
2. Size of data
3. Key variables - Demographic, Behavioral, Seasonal
4. Preprocessing needed

SSI Sales Data

Sales over Time

Analysis - Stutti

Analysis - Matt

Confusion Matrix

• 0: Non-high-revenue OPCO
• 1: High-revenue OPCO

Model Statistics

ROC Curve Analysis

Figure 1: ROC Curve for High Revenue Prediction

Feature Importance

Figure 2: Feature Importance for High Revenue Prediction

Analysis - Mika

Random Forest Model Summary

• The Random Forest model was trained to predict QuantityFulfilled using 100 records of sales data.
• The model used 8 predictor variables and 100 bootstrap iterations.
• Bootstrap validation was performed to assess model stability and performance.

Model Performance Metrics

• Original MSE: 800.89 units²
• RMSE: 28.30 units (shows average prediction error in original units)
• MAE: 13.09 units (shows average absolute prediction error)
• Bias: -204.04 (indicates the model tends to underestimate)
• Standard Error: 292.65 (shows variability in predictions)
• 95% Confidence Interval: (198.3, 1449.2) for MSE

Remarks

• The model predicts QuantityFulfilled with an average error of about 28 units (RMSE).
• Predictions are typically off by about 13 units (MAE).
• The model shows consistent negative bias (-204.04), suggesting systematic underestimation.
• A wide confidence interval (198.3 to 1449.2) indicates high variability in prediction accuracy.
• Numerical variables (qtyOrdered, TotalPrice) are substantially more important than categorical ones.

Conclusion

• High Performance & Versatile: Robust, accurate, handles noisy/high-dimensional data. [1]
• Ensemble Strength: Averaging over many trees; identifies key variables. [1]
• Challenges: Limited theoretical understanding; complex interpretation. [1]
• Future Focus: Enhance theory, increase interpretability, broaden applications. [1]